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Influence of presentation handout completeness on student learning 

in a physical therapy curriculum 

Erika Nelson-Wong 1 , Heidi Eigsti, Amy Hammerich, Nicole Ellison 

Abstract: Students and faculty have disparate opinions on how complete lecture 
materials should be to optimize learning. The purpose of this study was to 
investigate the influence of lecture handout completeness and content area on 
Doctor of Physical Therapy student recall/retention in foundation level courses. 

These findings suggest there may not be a best practice for lecture slide 
completeness and may be course/content dependent. In this study students 
performed better with complete handouts in physiology and research courses and 
with less complete handouts in biomechanics. 
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I. Introduction. 

There are consistent discrepancies between physical therapy student and faculty attitudes and 
beliefs regarding the completeness of lecture/presentation material provided to students. 
Anecdotal evidence gathered through informal surveys of students enrolled in the Doctor of 
Physical Therapy (DPT) program at Regis University suggest that students have a preference for 
access to complete presentation slides/handouts prior to and during the actual teaching sessions. 
Student comments on course and faculty evaluations also reflect this preference. Informal 
discussions that provided motivation for this study indicated that some students’ rationale for this 
preference is note taking during a presentation interferes with their ability to listen, and they are 
afraid of missing critical information. An informal survey of faculty in the School of Physical 
Therapy indicates a conflicting belief that note taking, rather than interfering with understanding, 
helps support student learning through the process of encoding information. In addition, some 
faculty members believe that providing incomplete presentation slides/handouts facilitates 
student engagement and encourages class participation. These anecdotal findings are in 
accordance with what has been published in the literature, where survey studies have shown 
dissonance in faculty/student beliefs surrounding handout completeness (Marsh & Sink, 2010; 
McClennan & Isaacs, 2008). Many investigators have found the majority of students prefer to 
have access to course presentation materials prior to classroom sessions (Marsh & Sink, 2010; 
Murphy, Gray, Straja, & Bogert, 2004; Yilmazel-Sahin & Oxford, 2010); however, few studies 
have investigated the influence of access to incomplete or complete handouts on learning 
outcomes (Achterberg, Duquiane, Huebbe, & Williams, 2007; Larson, 2009). Furthermore, few 
studies in this area have focused on graduate education (Yilmazel-Sahin & Oxford, 2010), and 
even fewer have been conducted in the area of physical therapy education. 
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A. Traditional Lecture Presentations in Physical Therapy Education. 

Traditional lecture presentations are not the primary instructional method employed in most 
physical therapy programs; however, some lecture presentations are frequently used (Willett, 
Sharp, & Smith, 2008), especially in foundation level courses, to introduce new concepts and 
provide background/basic knowledge that is necessary for students to advance to higher level 
application and synthesis activities. While traditional lecture may comprise a part of the content 
delivery in a course, it is important to understand that this may not be the most effective 
instructional strategy and should be used sparingly (Day, 1985; Hyland, 2010; Lake, 2001; 
Sellheim, 2006; Venglar & Theall, 2007; Wait et al., 2009; Wong & Driscoll, 2008). Many 
studies have been conducted investigating differences between teacher-centered (passive lecture) 
and student-centered (active learning) instructional approaches, and it is well accepted that an 
emphasis on student-centered learning is preferred (Day, 1985; Hyland, 2010; Lake, 2001; 
Sellheim, 2006; Venglar & Theall, 2007; Wait et al., 2009; Wong & Driscoll, 2008). However, 
traditional presentation style lectures are still often used, as one instructional component, 
particularly to present basic content in foundation science courses (Brahler, Quitadamo, & 
Johnson, 2002; Maring, Costello, & Plack, 2008; Sellheim, 2006). Ideally, traditional 
presentations comprise only a part of the content delivery, with case studies, facilitated group 
discussions, and hands on activities used in subsequent sessions to apply and reinforce concepts 
and promote critical thinking (Brahler et al., 2002; Chester, 2011; Zipp, 2010). Regardless of 
whether a classroom session is based around traditional lecture or more interactive activities, the 
use of visual materials such as PowerPoint slides has become ubiquitous. In the context of a 
traditional teaching presentation, the main purpose of PowerPoint slides is to guide and provide 
structure for the presentation, and ultimately clarify main concepts with the goal of improving 
student understanding (Achterberg et al., 2007; Brown & Manogue, 2001; Hill, Arford, Lubitow, 
& Smollin, 2012). In the Regis University School of Physical Therapy curriculum, PowerPoint 
slides are routinely used as described for the purpose of providing structure to a lecture 
presentation. 

B. The Role of Note Taking. 

Students rely heavily on notes taken during a class session to study from later, and typically 
instructors expect that students are taking notes during class sessions, especially when a 
traditional presentation format is being used. It has been suggested that note taking enhances 
learning through the process of encoding information and providing a record for students to refer 
to later during studying (DiVesta & Gray, 1972). It has also been shown that college students do 
not always take quality notes, and that as little as half of the critical information from a 
presentation may be recorded (Anderson & Armbruster, 1991). Different formats and methods 
for content delivery have been proposed to address the potentially competing goals of facilitating 
student engagement and encoding and allowing for an accurate and complete record of the 
content. Providing an outline or incomplete notes to supplement a teaching presentation has been 
suggested to help guide the student in note taking while facilitating encoding (Katayama & 
Robinson, 2000; Kiewra, 1985a). While providing a complete set of notes to students solves the 
problem of students not having an accurate external record for study (Kiewra, 1985a, 1985b), 
some faculty believe that this method may reinforce passive modes of learning and student 
disengagement. 
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C. Differences in Undergraduate, Graduate and Health Sciences Students. 

Most of the empirical research surrounding this topic has been done with a focus on 
undergraduate students, and therefore these findings may not generalize to physical therapy 
students enrolled in graduate programs. Yilmazel-Sahin and Oxford (2010) compared 
perceptions of graduate and undergraduate students in a teacher education program regarding 
completeness of handouts (downloadable presentation slides) accompanying traditional lecture 
presentations. It was determined that undergraduates took fewer notes than graduate students 
when handouts were provided, and also experienced greater stress when handouts were not 
provided. Graduate students reported using handouts to prepare for class in advance and stated 
this enabled them to engage with the course material more deeply. Graduate and undergraduate 
students alike reported both learning and note taking were improved when incomplete, guided 
outline handouts were provided compared to when complete handouts or no handouts were 
provided. The Yilmazel-Sahin and Oxford study was limited to the investigation of students’ 
perceptions and did not empirically test the impact on learning of different handout/slide formats. 

While studies comparing instructional methods are prevalent in the physical therapy 
education literature, no studies were found that investigated the influence of varying 
completeness levels of instructional materials on physical therapy student performance. One 
study conducted with dental students showed a preference for visual and read/write learning 
modalities, indicating higher satisfaction with incomplete Power Point presentations that 
necessitate note taking (Murphy et al., 2004). Unfortunately, this study did not evaluate the 
relative effectiveness of different slide presentation formats on student learning. 

D. Purpose and Hypotheses. 

The question of how to optimize teaching presentations to best facilitate student learning is 
certainly not new. However there remains a lack of empirical research, specific to the physical 
therapy student population, to guide educators in best practices for designing traditional teaching 
presentations and access to presentation handouts. The primary purpose of this study was to 
investigate whether presentation handout completeness influences learning in entry-level DPT 
students. Also of interest was whether handout completeness would impact immediate recall and 
longer-term retention of concepts. Finally, we aimed to investigate potential interactions between 
presentation handout completeness and course content. We hypothesized that less complete 
presentation handout formats would result in better learning, demonstrated on both immediate 
recall and longer-term retention quiz performance, than complete presentation handout formats. 
We also expected that the influence of presentation handout completeness on quiz performance 
would be similar between courses. 

II. Methods. 

This was a repeated measures, counterbalanced experimental design. Three School of Physical 
Therapy faculty members, each with primary responsibility for a foundation science course, as 
well as an Instructional Designer for the Regis University Rueckert-Hartman College of Health 
Care Professions participated in this study. The courses included: Management Applications of 
Physiology 1 (MAPI), Critical Inquiry (Cl), and Biomechanics (BIOM). These courses were 
selected for inclusion in the study as they are lower division courses taken during the first term 
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of the DPT program, contain foundation science level information, and the course structures all 
include traditional lecture presentations as a portion of the content delivery. The Regis 
University School of Physical Therapy curriculum follows the Normative Model of Physical 
Therapist Professional Education (A Normative Model of Physical Therapist Professional 
Education: Version 2004, 2004) and is therefore expected to be similar and comparable to other 
physical therapy programs. 

A. Subjects. 

A cohort of 66 entry-level DPT students beginning the first term in the School of Physical 
Therapy at Regis University were included in this study. Students in this program complete all of 
their coursework as a cohort, and take all classes together (no sectioning). The Regis University 
IRB determined the study to be exempt (assessment using standard educational practices) and 
therefore did not require consent to be obtained. Students were informed through standard 
statements in their course syllabi that Regis University faculty have an ongoing interest in 
investigating the effectiveness of instructional methods, and that quiz and test scores might be 
used for research purposes with all data de-identified. Signed learning contracts were obtained 
from each student, indicating agreement with course learning objectives and evaluative activities. 

B. Study Protocol. 

PowerPoint presentation slides were provided to the students prior to the class session as 
downloadable handouts, and were organized in the following manner. The three levels of 
handout completeness were: complete, fill in the blank, and incomplete. Complete handouts were 
defined as PowerPoint slides that contained all concepts for the presentation with no new 
information introduced verbally by the instructor, these handouts exactly mirrored the 
presentation slides. Theoretically, note taking would not be necessary and students would be free 
to listen to the instructor without the distraction of note taking. Fill-in-the-blank handouts were 
defined as PowerPoint slides that were 75% complete, with selected blanks, indicated by an 
underline, where students would need to fill in a missing word or concept. Incomplete handouts 
were defined as PowerPoint slides that contained 50% of the information for the presentation, 
necessitating note taking to fill in content gaps and expand upon concepts. These handouts were 
typically constructed as bullet-point headings, requiring the students to fill in additional 
information. Representative examples of the three-slide/handout formats are included in 
Appendix 1. All handouts of presentation slides were posted on a web-based learning 
management system (ANGEL) a minimum of 24 hours prior to the teaching session per standard 
SPT policy. Additional readings (journal papers and/or textbook) were recommended for each 
class session in the respective course syllabus. This study did not track student adherence with 
completing the additional readings. 

Each of the three faculty members selected three sessions within their respective course 
for inclusion in the study, and provided a standard 50-minute presentation for each session, one 
each of the three PowerPoint formats (complete, fill in the blank and incomplete) for a total of 
three presentations in each of the three courses. The format for the first presentation was 
randomly assigned to each course, and subsequent order was counterbalanced to ensure that each 
course used a different order for the three formats to minimize order effects. Table 1 contains the 
topic area and presentation format that was used for each course. 
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Table 1. Topic areas for each course and counterbalanced presentation order. 


Management 
Application in 
Physiology I (MAP 1) 

Critical Inquiry (Cl) 

Biomechanics (BIOM) 

Complete 

Physiologic Responses to 
Exercise 

Fill in the Blank 
Evidence Based Practice 

Incomplete, 

Bone Biomechanics 

Incomplete 

Metabolism 

Complete 

Fevels of Measurement 

Fill in the Blank, 

Hip Biomechanics 

Fill in the Blank 
Neuromuscular 
Physiology 

Incomplete 
Comparison Between 
Means (/-test) 

Complete, 

Thoracic Spine Biomechanics 


Any information that was missing in the handout was provided during the class session 
providing the students the opportunity to take notes during the session. The Instructional 
Designer reviewed the handouts and slides prior to each session to ensure they met the 
criteria/definitions for each given format and to attempt to minimize disparity in handout/slide 
construction between faculty members. 

The dependent variables for this study were performance on immediate recall and longer- 
term retention quizzes (percent correct). Quizzes consisted of five to seven multiple choice 
questions administered with fill in the bubble answer sheets. Quiz questions were written at the 
comprehension, application, analysis and synthesis levels, similar to expectations for 
examination questions within each of the three courses (Appendix 2). Immediate recall and 
longer-term retention quizzes were given for each slide format and consisted of the same 
questions presented in a different order. Immediate recall quizzes were given directly following 
the presentation, and longer-term retention quizzes were given at the start of the next course 
session (between two and seven days later). Quizzes were reviewed in advance by the 
Instructional Designer to help ensure the questions for each course and slide format were similar 
in design and level of difficulty. The timing of quizzes was not disclosed to the students in an 
attempt to prevent confounding effects of differences in advance preparation. The students were 
told that the quizzes were for the sole purpose of faculty feedback on student learning and were 
not counted towards the student’s course grade. 

Quiz scores (percent correct) were entered into a 3x3x2 within factor ANOVA, with 
factors of slide format (three levels: complete, fill in the blank, incomplete), course (three levels: 
MAPI, Cl, BIOM) and quiz timing (two levels: recall, retention). Students who were absent 
from a class and missed a quiz had their data excluded from the analysis. Significance criterion 
was set at a = .05. Pairwise comparisons using /-tests (with Bonferroni adjustment for multiple 
comparisons) were used for post hoc analyses when necessary. Statistical analyses were 
performed using SPSS version 18.0 (IBM, Amonk, NY). 

III. Results. 

Of the 66 students in the cohort, 14 were excluded due to class absences, leaving a sample of 52 
students. The number of students that were absent for a given session varied from zero to three, 
with an average absentee rate of 2.3%. The student cohort was 45% male. 70% were in the 20-25 
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year age bracket, 23% in the 26-30 year age bracket, and the remaining 7% were older than 30 
years. Summary descriptive statistics for recall/retention quiz performance (% correct) for each 
of the three slide formats in the three courses are provided in Table 2. 


Table 2. Recall and retention quiz results for each slide format for the 3 courses. 


Course 

Handout Format 

Recall % Correct 
Mean (SD) 

Retention % 
Correct 

Mean (SD) 

Management 
Applications in 

Complete 

86.0(14.3) 

88.5 (12.7) 

Physiology 1 

Fill in the Blank 

72.5 (17.4) 

72.8(21.4) 

(MAPI) 

Incomplete 

76.9(15.8) 

80.2(15.8) 

Critical Inquiry 

Complete 

88.1 (15.0) 

87.3 (16.8) 

(Cl) 

Fill in the Blank 

60.8(16.3) 

66.2 (16.6) 


Incomplete 

48.1 (23.5) 

70.1 (24.6) 

Biomechanics 

Complete 

75.3 (21.0) 

76.0 (21.5) 

(BIOM) 

Fill in the Blank 

88.1 (13.7) 

86.2 (15.4) 


Incomplete 

88.5 (15.7) 

86.9(14.5) 


There was a significant 3-way interaction between handout format, course, and quiz 
timing (F 4,204 = 9.652,/? < .001). There were also significant 2-way interactions between course 
and handout format {F 4,204 = 42.3,/? < .001) and course and quiz timing (F 2,102 = 12.6, p < .001). 
To investigate the interactions of interest, nine pairwise comparisons were made using paired t- 
tests for each. Students performed best with the complete slide format, in both immediate recall 
and longer-term retention, in the MAPI and Cl courses, while they performed worst with this 
format for the BIOM course. Student performance on immediate recall was significantly 
different in all three courses for fill-in-the-blank; however, longer-term retention scores were 
similar between MAPI and Cl. Immediate recall quiz scores were also different between all 
three courses for the incomplete handout format, with the only difference persisting on longer- 
term retention being between Cl and BIOM. Immediate recall and longer-term retention quiz 
scores were stable with all three handout formats for both MAPI and BIOM, as well as for the 
complete handout format for CL In the Cl course, students improved in their longer-term 
retention quiz scores for both the fill-in-the-blank and incomplete handout conditions. The 3-way 
interaction is shown in Figure 1. For clarity, Figures 2-4 depict the findings by each course. 
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-o- MAPI Recall -^0—MAPI Retention ••■□••Cl Recall 

—□—Cl Retention -■©-•BIOM Recall — #-BIOM Retention 

Figure 1. The 3-way interaction between course (Management Applications in Physiology 1 [MAPI], 
Critical Inquiry [Cl], Biomechanics [BIOM]), handout/slide format (Complete, Fill in Blank, 
Incomplete) and quiz timing (Recall, Retention). 



■ Recall Quiz El Retention Quiz 


Figure 2. Recall and retention quiz results between handout/slide formats for the Biomechanics 
(BIOM) course. 
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* p< .001 



Complete Fill in Blank 

n Recall Quiz □ Retention Quiz 


Incomplete 


Figure 3. Recall and retention quiz results between handout/slide formats for the Management 
Applications in Physiology 1 (MAPI) course. 



Figure 4. Recall and retention quiz results between handout/slide formats for the Critical Inquiry 
(Cl) course. 
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IV. Discussion and Conclusion. 

The primary purpose of this study was to investigate whether presentation handout completeness 
influenced learning, assessed by performance on immediate recall and longer-term retention 
quizzes, in entry-level DPT students with an additional consideration for the influence of 
differing course subjects. Our hypothesis that students would perform in similar ways regardless 
of the course, with less complete handout formats being optimal, was rejected as students 
showed reverse patterns of performance between BIOM and MAP1/CI courses. 

It was a surprise in the current study to find differences in student recall and retention 
between course content areas and handout completeness. In a similarly designed study, Larson 
(2009) found the highest student recall occurred when incomplete handouts were used, and the 
lowest recall with complete handouts. These findings were in a different student population, 
undergraduate business students, and the study was conducted within the context of a single 
course topic area, learning negotiation skills. The current findings suggest that there may not be a 
single best practice for handout completeness, and it may vary by type of course or by instructor. 
A major limitation of the current study design is that differences due to individual instructor 
presentation styles cannot be separated from differences due to course content. Although efforts 
were made to ensure that presentation slides and corresponding handout construction were 
similar between courses, it is certain that presentation delivery was not identical. Additionally, 
these findings are from one student cohort at a single institution, and while physical therapy 
students and curricula between institutions are expected to be similar, differences cannot be ruled 
out which may limit the generalizability of these findings. Another limitation of the study was 
the length of the quizzes, with only five to seven questions on each quiz. Other studies that have 
used quizzes to assess learning of concepts have used a similar number of items as this study 
(FitzPatrick, Finn, & Campisi, 2011; Larson, 2009). Because each quiz only covered concepts 
from a single 50-minute lecture, the authors felt that the number of items was appropriate for the 
amount of information covered. The influence of student learning style preferences is not 
included in the current findings. These data, as well as qualitative data obtained from student and 
faculty focus groups and surveys, are being analyzed separately as a second aim to this overall 
research project. 

The topic areas that were selected by the instructors for inclusion in the study were 
mainly ‘concept based’, so although BIOM and Cl (a course that includes statistics) were 
included in the study, those presentations did not emphasize equations or mathematical problems. 
However, concepts presented in MAPI and Cl (evidence based practice, physiological 
responses) may have been more conceptually abstract than the BIOM content (biomechanics of 
tissues and specific body regions), which tended to be very literal and concrete, and this could 
have had an influence on student response. It was encouraging to find that immediate recall and 
longer-term retention performance were stable for MAPI and BIOM courses, regardless of the 
handout format used. Additionally, complete handouts which appeared to be optimal for students 
in the Cl course, resulted in stability between immediate recall and longer-term retention quiz 
scores. It is possible that students in Cl recognized they had performed very poorly on the 
immediate recall quiz with the incomplete handouts, and engaged in additional review/study of 
the concepts independently prior to the subsequent class session and retention quiz. These 
findings suggest that once instructors determine the most effective presentation and handout 
format for their particular course content, they can be fairly confident that student learning can 
be optimized in both immediate recall and retention. 
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Health science educators who teach foundation level curricular content that may be 
considered to be more abstract or conceptual might find that student learning is enhanced when 
presentation handout materials have a higher level of completeness. Educators who teach more 
concrete, problem-based content might improve student performance by providing less complete 
presentation materials. Given that it is impossible to parse out the influence of instructor style 
from this study, educators are encouraged to be aware that there may not be a single best practice 
regarding completeness of presentation materials provided to students a priori. Student learning 
in response to different presentation and handout formats can be easily assessed through pop- 
quizzes as a method of ascertaining what the best practice for an individual instructor and student 
group may be. This can be accomplished by developing handout materials that differ in level of 
completeness for different teaching sessions (or for sub-sections within the same session), 
followed by a short quiz over the content that was covered. Comparisons can then be made to 
provide insight into the most effective approach for the particular course, student cohort, and 
instructor combination. Because the immediate recall to longer-term retention performance was 
fairly stable within each course in this sample, it may not be necessary to assess both conditions 
to determine the best approach. 

In conclusion, findings from this study provide evidence that physical therapy student 
learning varies with level of completeness of supplemental handouts as well as course 
content/instructor. Incomplete presentation handouts where students are required to fill in 
missing information through note-taking may be best utilized in concrete, problem-based courses 
such as biomechanics/kinesiology. Complete presentation handouts that allow students to write 
less and therefore be more engaged in listening may be best for courses that include a greater 
degree of abstract or conceptual topics such as physiology/health and wellness applications and 
evidence-based practice/research methods/statistics. Ultimately, educators should examine the 
effect of their own presentation slide and related handout format on student learning, particularly 
if they engage in teaching multiple content areas. 
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Appendices 

Appendix 1. Representative Examples of the Three Handout Formats 

Example Fill in the Blank Slide/Handout - Biomechanics (BIOM), Hip Biomechanics 

Angle of Torsion 

* Transverse Plane 

* Long axis of femoral head to 
transverse axis of femoral 
condyles 

* Adults - typically_° 

* Can range from_° 

* Anteversion - angle >_° 

* _rotation of femur 

* Retroversion - angle <_° 

* _rotation of femur 



Example Incomplete Slide/Handout - Management Applications in Physiology I (MAPI), 
Metabolism 


From an Energy Transfer Standpoint: 

What Limits Physical Activity? 


• Exercise intensity and duration depend 
on the ability of the cells to 


• What are some other limiting factors? 
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Example Complete Slide/Handout - Critical Inquiry (Cl), Levels of Measurement 


Measurement 


Nominal 

• Least sensitive 

• Naming categorizing 
(grouping) data 

• Have no relationship with 
each other 

• Ex: Name of states 


Ordinal 

• Add magnitude to the 
categorization 

• Ranking or order 

• Doesn't indicate to what 
degree the/ are ranked 

• Ex: 

Ptofesso r:Assist ant, Associate, 
Full 

Manual Muscle Testing 


Appendix 2. Example Quiz Questions for Each of the Three Courses 

Biomechanics (BIOM) 

A retroverted hip will result in a_femur during gait, in order to keep the 

femoral head in the acetabular cavity. 


a. abducted 

b. adducted 

c. internally rotated 

d. externally rotated 
Management Applications in Physiology I (MAPI) 

What are the 2 determining factors of fuel source in metabolism 

a. Intensity and skill 

b. Genetics and training 

c. Time of activity and skill 

d. Intensity and time of activity 


Critical Inquiry (Cl) 

Researchers investigated whether the type of stretching (dynamic vs. static) and the 
presence of warm up (warm-up vs no warm up) effects the strength of the hip 
musculature (Manual Muscle Test) in female collegiate soccer players. What is the level 
of measurement for the dependent variable? 


a. 

nominal 

b. 

ordinal 

c. 

interval 

d. 

ratio 
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